Increse duckdb retries when operating on parquet s3 - #8903
Conversation
Signed-off-by: Robert Kruszewski <github@robertk.io>
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: Vortex queries 📖Verdict: No clear signal (medium confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.239x ❌, 0↑ 2↓)
datafusion / parquet (0.990x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.961x ➖, 0↑ 0↓)
duckdb / parquet (1.003x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: PolarSignals Profiling 📖Vortex (geomean): 1.088x ➖ How to read Verdict and Engines
datafusion / vortex-file-compressed (1.088x ➖, 0↑ 6↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.074x ➖, 0↑ 2↓)
datafusion / vortex-compact (1.042x ➖, 0↑ 2↓)
datafusion / parquet (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.010x ➖, 0↑ 1↓)
duckdb / vortex-compact (1.017x ➖, 0↑ 1↓)
duckdb / parquet (1.024x ➖, 0↑ 1↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.997x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.995x ➖, 0↑ 0↓)
datafusion / parquet (1.009x ➖, 1↑ 2↓)
datafusion / arrow (1.003x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (0.992x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.992x ➖, 0↑ 0↓)
duckdb / parquet (0.992x ➖, 1↑ 1↓)
duckdb / duckdb (0.991x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.036x ➖, 0↑ 5↓)
datafusion / vortex-compact (1.020x ➖, 0↑ 1↓)
datafusion / parquet (1.028x ➖, 1↑ 9↓)
duckdb / vortex-file-compressed (1.015x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.015x ➖, 1↑ 3↓)
duckdb / parquet (1.017x ➖, 0↑ 3↓)
duckdb / duckdb (1.013x ➖, 2↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb S3 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.987x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.991x ➖, 0↑ 0↓)
datafusion / parquet (1.025x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.036x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.021x ➖, 0↑ 0↓)
duckdb / parquet (1.044x ➖, 0↑ 0↓)
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.028x ➖, 1↑ 1↓)
datafusion / parquet (0.989x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.954x ➖, 1↑ 0↓)
duckdb / parquet (1.006x ➖, 0↑ 0↓)
duckdb / duckdb (0.993x ➖, 0↑ 0↓)
File Size Changes (201 files changed, -0.0% overall, 98↑ 103↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed (0.980x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.993x ➖, 0↑ 0↓)
duckdb / parquet (0.980x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.013x ➖, 1↑ 3↓)
datafusion / parquet (1.034x ➖, 0↑ 3↓)
duckdb / vortex-file-compressed (0.970x ➖, 8↑ 2↓)
duckdb / parquet (1.004x ➖, 0↑ 1↓)
duckdb / duckdb (0.997x ➖, 0↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.023x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.029x ➖, 0↑ 1↓)
datafusion / parquet (1.005x ➖, 0↑ 0↓)
datafusion / arrow (1.012x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.003x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.014x ➖, 0↑ 0↓)
duckdb / parquet (1.011x ➖, 0↑ 0↓)
duckdb / duckdb (1.006x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.098x ➖, 0↑ 3↓)
datafusion / vortex-compact (1.121x ➖, 0↑ 0↓)
datafusion / parquet (1.080x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (1.047x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.045x ➖, 0↑ 0↓)
duckdb / parquet (1.037x ➖, 0↑ 0↓)
|
|
Let run the benchmarks a few times before merging |
Benchmarks: Appian on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.988x ➖, 0↑ 0↓)
datafusion / parquet (1.017x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (0.985x ➖, 0↑ 0↓)
duckdb / parquet (0.984x ➖, 0↑ 0↓)
duckdb / duckdb (0.985x ➖, 0↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: Random Access 📖Vortex (geomean): 1.030x ➖ How to read Verdict and Engines
unknown / unknown (1.022x ➖, 1↑ 5↓)
|
Benchmarks: TPC-H SF=10 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (0.985x ➖, 1↑ 1↓)
datafusion / vortex-compact (1.036x ➖, 0↑ 0↓)
datafusion / parquet (1.085x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (1.036x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.029x ➖, 0↑ 0↓)
duckdb / parquet (1.072x ➖, 0↑ 0↓)
|
Benchmarks: Compression 📖Vortex (geomean): 1.000x ➖ How to read Verdict and Engines
unknown / unknown (0.930x ➖, 54↑ 2↓)
|
This roughly matches object_store behaviour